Repository navigation
chore(release): v0.6.0 - #335
Merged
Merged
Conversation
chore: merge main into next after v0.5.0
berth notify had no Windows side: berthd runs as a service in session 0, which has no desktop, and notify-send is Linux only. On Windows, berth notify is a notification-area icon in the user's session, in plain Win32 through x/sys/windows with no new dependency. Its balloons are Windows' own toasts, and a click opens the harbor. The tooltip is the latest event. The menu opens the harbor, turns the toasts off and on, lists the last 10 events and quits. The icon comes back when Explorer restarts. berth service install --notify starts it at logon from the user's Run key, and uninstall removes the entry.
feat(notify): Windows tray with native toasts
A ComfyUI job that calls an LLM in the middle deadlocked under Berth. The LLM's drain of ComfyUI waits for the job boundary, the job waits for the LLM, and the LLM waits for the card (#299). POST /v1/berth/yield {"engine":"comfyui","seconds":120} tells ComfyUI's berth run that the running job freed its models in-process and is at a safe point. Until the yield ends or runs out, a drain treats the job as at its boundary, so the LLM's drain parks ComfyUI as it would between jobs. {"end":true} resumes ComfyUI and answers once it holds the card again. The yield moves only the boundary. The park still counts what the card read shows free, a park that frees nothing still falls back to a stop, and a yield is a deadline, so a node that dies mid-yield cannot leave it on.
…t-orphan-wrapper # Conflicts: # CHANGELOG.md
…apper chore: merge main into next (#306)
Two bugs found in review of the yield: - The end answered before ComfyUI had the card back. berthd queued the parked lease, so the gateway's own resume was refused as not parked, and its probe of ComfyUI's queue answered, as a parked ComfyUI always does. The job's next node could load onto memory the LLM still held. The end now waits for lease.resumed when berthd says it queued a parked engine. - A lease on several cards took the yield and its run never heard it, so the call reported success and the deadlock stayed. A share cannot park yet, so the yield is refused with that reason.
feat(gateway): a ComfyUI job can yield the card to the LLM it calls
…nnot fit beside A whole-card waiter was granted beside any parked residual under residual_max_gib. On aphotic, Dirk-27B holds 23.4 GiB of a 24 GiB card and was granted beside a kept-up ComfyUI's 768 MiB context, where it cannot load. A wrapper script stopped ComfyUI behind Berth's back to get around it. The footprint table now keeps the most VRAM each engine name has held (HotPeak, on the name's entry: a llama-swap group's peak is its largest model's). A whole-card waiter goes beside parked engines only when that peak fits beside their residual; otherwise they are evicted first, least recently parked first, as a residual over the max already is. An engine never measured is granted as before.
fix(broker): a model that fills the card evicts a parked engine it cannot fit beside
…s beside a parked engine The live test of #310 on aphotic: Dirk-27B held 23710 MiB alone and served. Beside a parked ComfyUI's 456 MiB it was granted, reached 24147 MiB on the 24564 MiB card and died loading, though its peak plus the residual left 398 MiB. The 2 s card read misses the last allocations of a load, so the fit check now keeps 512 MiB past the measured peak.
fix(broker): keep 512 MiB of load headroom when a full-card model fits beside a parked engine
…_size learn_size = true on an engine entry, or berth run --learn-size, sizes the lease from the most the engine has been measured holding plus the 512 MiB load headroom #312 measured, in place of the whole card. An engine with no measured run takes the whole card. The grant line names the size it used. A typed needs_mib, several cards and whole_machine are refused with it. Refs #311
feat(broker): size a lease from the engine's measured peak with learn_size
berth status gains a SCHEDULE table and the harbor a SCHEDULES panel: each engine that has a window, its hours, whether they are open, and how long until they close or open, read on the node's clock. An engine shows before its window opens, and a berth run --window lease shows its own hours. The status JSON carries the rows as schedules, and each lease its window.
berth train runs CMD as a training lease: a pinned hold on the whole of each card it names, with no wait deadline. While it holds a card or waits for it, berthd refuses every other request on that card at once, and refuses the waiters already queued there, so gateway requests, berth run, llama-swap models, scheduled jobs and berth hold get an answer right away instead of waiting out their timeout. A second training waits in line and starts when the first one ends. Other cards keep serving. A game or hold already on the card is not drained. berth train refuses to start without berthd, and a training re-attaches as a training after a berthd restart.
The harbor screenshots show the current tabs, SWITCH included, all from the demo fixture. How it works names berth train beside games and the gateway's per-card URLs. The README adds a month of counts from the reference workstation, and assets/social-preview.png is a 1280x640 social preview for the repository settings.
feat(broker): berth train locks its cards for a training run
docs: refresh the harbor screenshots, How it works and the README
# Conflicts: # internal/broker/status.go
feat(status): show every engine's window in berth status and the harbor
A berth train run records itself as a training when it loses berthd. With berthd down, berth run refuses to start an engine while a live training record exists, instead of starting it without a lease. A berthd that starts while a training runs treats its record as the lock: it refuses every other request until the training re-attaches, with no time limit, where other engines get the 15 second re-attach window.
feat(broker): a training keeps its lock while berthd is away
berth train --exclusive puts the granted cards in exclusive-process compute mode after the grant and before the command starts, so a program that opens CUDA without Berth cannot open them, and puts them back to the default mode when the run ends. It refuses to start when another compute process is on the card, when no sudo rule lets nvidia-smi -c run without a password, and on a card with no NVIDIA UUID. Linux only.
feat(runner): berth train --exclusive locks the card in the driver
By default llama-swap runs one model at a time and unloads the old model itself when a request names another. Berth sees a voluntary release, so min_turn, the engine ranking and the juggle guard never decide between two llama-swap models. On aphotic two clients made the card swap 82 to 135 times an hour this way. berth wire llama-swap now lists each group (or the default group) where llama-swap swaps two or more wired models, in the human output and as "swapping" in the JSON. contrib/INTEGRATION.md shows the swap: false group and a healthCheckTimeout longer than min_turn plus the load. wire also finds a berth run line with a flag before --as (berth run --min-turn 5m --as qwen ...). It took such a model for unwired, and --write added a second berth run in front of the first.
feat(wire): name the llama-swap groups that swap wired models themselves
feat(fit): berth fit NAME shows where a request would run and why
…iter Berth grants waiters by priority, then arrival, so many devices can send work to one card with nobody sorting it. berth queue is for when the operator wants a different order. With no arguments it lists the waiters in grant order with their lease IDs. bump moves one to the front of its priority class (a bumpedN tiebreak in sortQueue, latest bump first), prio moves it to batch, interactive or foreground, and drop refuses it the way the training lock does. A hold or a training cannot be changed here. Each change is a lease.queue_override event. A new OpQueue goes through the node to every card the waiter waits on.
feat(queue): berth queue lets the operator bump, re-rank or drop a waiter
…d device The SWITCH tab lists the waiters in grant order from the node's status. A paired device gets FIRST (bump), a priority picker and DROP on each one, except for a training or a hold, which the broker refuses to change. POST /v1/board/queue goes from the harbor straight to berthd's socket after the pairing check. There is no gateway route on purpose: agents hold the gateway's API key, and a queue change is the operator's.
feat(harbor): a QUEUE panel with FIRST, priority and DROP for a paired device
…state berth contract showed a llama-swap model as interactive with no minimum turn, because those terms sit on its berth run line and no berth.toml entry declares them. The broker now keeps the priority and minimum turn of the last run under the model's own name, and the contract reads them from there. The last peak falls back to the most a run under that name held, since the last-run figure sits on the group's entry. berth fit refused a llama-swap model that no [[engine]] names or serves. It now finds the model the way the gateway does, on a live engine whose /v1/models lists it. The gateway's list read moves to gateway.ListModels so both use one call. berth fit also told an engine already on the card to drain itself. A new running path ranks first, in the hot tier, for an engine whose own lease, or a lease of a model in its group, is active.
fix(fit,contract): read a llama-swap model's terms, list and running state
A non-streaming request whose start timed out got a bare 503 with Retry-After 30, even while its lease stood in line for the card. The gateway now reads the lease's place in line from berthd when the start times out, and the 503 carries X-Berth-Queue-Position and, when the broker has measured the starts ahead, X-Berth-Start-Estimate: N; lower-bound. Retry-After is the larger of the estimate and 30 s. The broker's estimate counts only the measured starts ahead in the queue and not the holder's time left, so the header marks it as a lower bound instead of passing it off as a promise.
feat(gateway): a not_ready 503 says the lease's place in line
…wntime berthd keeps a schedule per card, up to 14 days ahead, in schedule.json. A run booking is an engine's turn: its waiter goes first, runs at the booking's priority, wins a tie against an unbooked holder, and keeps the card against everything but a hold until the booking ends. A priority booking moves an engine to another class for set hours. Downtime takes no new work and drains holders at the start; holds and trainings stay. Hours nobody booked run by priority, then arrival, as before. berth schedule shows what holds each card, the queue and the bookings, and books, moves and cancels them. The gateway's /v1/berth/schedule lets agents read the scheduler and book their own runs. The harbor gains a SCHEDULE tab with a 7-day board of 24-hour tracks, and a paired device books from it through berthd's socket.
feat(schedule): book each card's hours: runs, priority changes and downtime
berth run now sends how long its engine has held the card since its last grant or resume, and the new berthd counts the minimum turn from then. Before, every daemon restart reset each holder's min_turn, so a deploy kept waiters out for a full turn again.
fix(runs): a re-attach goes on with the turn it was in
berthd appends each event to events.jsonl in the state directory and reads the last 1000 back at start, so sequence numbers and lease numbers go on after a restart and each start is a node.started event. berth why ID|ENGINE prints one lease's events as a story: the waits, the grant, each drain and who it was for, the release, and the restarts it lived through.
berth engine show --help printed "berth show", and a bad flag pointed at "berth show -h". The context now carries the command path as typed.
feat(events): keep the event stream across restarts, and berth why ID
fix(cli): a subcommand's help names its group
…nswers identical Thinking models spent the bench's 32 tokens inside their reasoning, so every sample's text was empty and OUTPUT IDENTICAL printed yes. The bench now reads reasoning_content and reasoning, compares them with the answer, prints n/a (empty) when all runs are empty, defaults to 256 tokens (-max-tokens) and has -no-think.
TestGatewayResumesAParkedEngine and the yield test left their sleep 30 engine and runner.Run going. At cleanup the daemon closed, Run wrote its unprotected-run record into the temp dir being removed, and CI failed with "directory not empty" (#332's check job). runUntilEnd kills the engine's group and waits for Run.
A resume cleared the parked flag before it called the engine's load, so a drain in that window called park on an engine still loading: the engine answered 409 and the run stopped it (#275). The message loop now marks the wake before the resume goroutine starts, and drain waits for it to end.
test(gateway): end each test's berth run before its temp dir goes
fix(bench): the swap bench compares reasoning and never calls empty answers identical
…or-resume # Conflicts: # CHANGELOG.md
fix(runner): a drain waits for a resume in flight
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merges
nextintomainfor v0.6.0 and dates the CHANGELOG section.What is in it
Each card gets a schedule (
berth schedule,/v1/berth/schedule, the harbor's SCHEDULE tab).berth trainruns trainings nothing automated can interrupt.berth fit,berth contract,berth queueandberth whyexplain the card, and the event log survives restarts.learn_size, the ComfyUI yield (#299), the Windows tray (#186), and fixes for restarts and the resume race (#275). The full list is the CHANGELOG's v0.6.0 section.Docs
berth why,fitandcontractunder what Berth sees.v0.6.0-rc.1.mkdocs build --strictpasses.Before merge
v0.6.0-rc.1is published as a pre-release from the samenextcommit.proof.shfull is green.